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PREFACE 


This  documented  briefing  outlines  research  undertaken  in  support  of 
emerging  Air  Force  employment  strategies  associated  with  Expeditionary 
Aerospace  Forces  (EAFs).  Although  much  work  has  yet  to  be 
accomplished  in  defining  and  preparing  Air  Force  units  for  meeting  these 
new  responsibilities,  it  is  clear  that  EAF  concepts  will  play  a  central  role  in 
the  future  Air  Force.  EAF  concepts  turn  on  the  premise  that  rapidly 
deployable,  immediately  employable,  highly  effective,  and  flexible  air  and 
space  force  packages  can  serve  the  same  strategic  role  as  a  permanent 
forward  presence  in  deterring  aggression  and,  if  necessary,  responding  to 
aggressive  acts.  The  success  of  the  EAF  will  to  a  great  extent  depend  on 
the  effectiveness  and  efficiency  of  the  support  system  that  undergirds 
flying  operations.  The  Air  Force  has  named  such  a  support  system  one  of 
its  six  necessary  core  competencies  and  has  labeled  it  the  Agile  Combat 
Support  (ACS)  system. 

The  efficiency  and  effectiveness  of  ACS  are  affected  by  decisions  made 
across  programming  and  budgeting  time  horizons.  Far-term  ACS 
decisions  affect  future  support  structures  required  to  meet  operational 
requirements  with  future  force  mixes.  Mid-term  ACS  decisions  affect  the 
design,  development,  and  evolution  of  the  support  infrastructure  for 
meeting  operational  requirements  within  the  programming  and 
budgeting  time  horizons.  Near-term  decisions  affect  where,  when,  and 
how  existing  resources  are  employed.  Across  this  time  spectrum,  logistics 
requirements  can  be  satisfied  in  a  variety  of  ways,  each  with  different 
costs,  flexibility,  response  times,  and  risks. 

This  documented  briefing  discusses  research  supporting  RAND's  ongoing 
assessment  of  intermediate-level  support  options  for  the  electronic 
countermeasure  (ECM)  pod  system.  This  particular  study  addresses  the 
usefulness  of  the  Reliability,  Availability,  and  Maintainability  of  Pods 
(RAMPOD)  database  as  an  analytical  tool  in  support  of  the  ECM  pod 
study  that  is  part  of  ACS  efforts. 

Our  research  shows  that  RAMPOD  already  has  a  great  deal  of  useful 
information  both  in  the  on-line  database  reports  and  recorded  in  the  data¬ 
warehousing  system.  With  some  modifications  to  the  current  data 
presentation  format  and  the  incorporation  of  warehoused  data  into  the 
web  query  tool,  RAMPOD  can  become  valuable  both  for  operational 
analyses,  such  as  process  performance  monitoring,  and  for  strategic 
planning  activities. 
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The  research  addressed  in  this  report  was  conducted  in  the  Resource 
Management  Program  of  Project  AIR  FORCE  as  one  element  of  a  project 
entitled  "Evaluating  Agile  Combat  Support  Options  for  Implementing  the 
Expeditionary  Aerospace  Force  (EAF)."  This  project  was  sponsored  by 
the  Air  Force  Deputy  Chief  of  Staff  for  Installations  and  Logistics  (AF/IL). 
This  report  should  be  of  interest  to  logisticians,  operators,  and  mobility 
planners  throughout  the  Department  of  Defense,  especially  those  in  the 
Air  Force.  Research  for  this  document  was  completed  in  July  2000. 

For  further  information,  please  contact  the  lead  author,  Patrick  Mills,  at 
(310)  393-0411,  ext.  7983,  or  Patrick_Mills@rand.org. 


PROJECT  AIRFORCE 

Project  AIR  FORCE,  a  division  of  RAND,  is  the  Air  Force  Federally 
Funded  Research  and  Development  Center  (FFRDC)  for  studies  and 
analyses.  It  provides  the  Air  Force  with  independent  analyses  of  policy 
alternatives  affecting  the  development,  employment,  combat  readiness, 
and  support  of  current  and  future  aerospace  forces.  Research  is 
performed  in  four  programs:  Aerospace  Force  Development;  Manpower, 
Personnel,  and  Training;  Resource  Management;  and  Strategy  and 
Doctrine. 
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Supporting  Expeditionary  Aerospace  Forces: 
Evaluation  of  the  RAMPOD  Database 


Prepared  for  Patricia  R.  Martin,  Deputy  Director, 
Electronic  Warfare  Managment,  WR-ALC/LN 


INTRODUCTION 

Although  its  definition  has  not  been  finalized,  the  Expeditionary 
Aerospace  Force  (EAF)  concept,  which  organizes  the  Air  Force  to 
respond  rapidly  to  national  security  threats  with  tailored  sustainable 
force,  is  certain  to  play  a  central  role  in  the  future  of  the  U.S.  Air  Force. 
Several  RAND  reports  have  outlined  the  importance  of  Agile  Combat 
Support  (ACS)  in  meeting  the  rapid  deployment  and  immediate 
employment  requirements  associated  with  the  goals  of  the  EAF  concept.1 
Tripp  et  al.  (2000)  presented  an  analytical  framework  to  guide  the  design 
and  evaluation  of  ACS  systems.  In  one  of  a  series  of  follow-up  studies 
that  use  this  analytical  framework,  researchers  examine  how  alternative 
maintenance  support  concepts  for  electronic  countermeasure  (ECM) 
pods  can  improve  ACS  for  the  EAF.  The  documented  briefing  here  is  a 
subtask  of  the  ECM  pod  analysis.  It  focuses  on  the  utility  of  the 
Reliability,  Availability,  and  Maintainability  of  Pods  (RAMPOD) 
database  as  an  analytical  tool,  particularly  as  it  applies  to  the  broader 
ECM  pod  study. 


'For  a  more  detailed  discussion  of  this  work,  see  Tripp  et  al.  (2000). 
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Project  Background 


•  ECM  Pod  Project 

-  RAMPOD  is  one  of  the  tools  we  plan  to  use  for  the 
analysis  of  multiple  ECM  pod  support  options 

•  RAMPOD  Evaluation 

-  Requested  by  Patricia  Martin,  Deputy  Director,  Electronic 
Warfare  Management,  to  evaluate  RAMPOD  database  as 
an  analytical  tool 

•  RAND  Focus 

-  184  long:  269  pods  (sample  data/charts) 

-  184  short:  628  pods 

-  131  three-band:  481  pods 

RAND  Project  AIR  FORCE  m 


The  ECM  pod  intermediate-level  maintenance  (ILM)  project  was 
undertaken  as  a  subtask  within  RAND's  ongoing  ACS  work,  sponsored 
by  AF/IL.  The  study  addresses  support  structure  alternatives  for 
meeting  demands  for  ECM  pods  across  the  spectrum  of  EAF  optional 
requirements  from  major  theater  wars  to  peacetime  operations. 

The  RAMPOD  database,  a  logistics  engineering  support  system  for 
electronic  combat  pods  and  integrated  systems,  is  one  of  the  tools  we 
plan  to  use  in  the  ECM  pod  study.  As  a  subtask  within  the  ECM  study, 
we  were  asked  by  Patty  Martin  to  evaluate  the  usefulness  of  RAMPOD  as 
an  analytical  tool.  This  report  summarizes  our  analysis. 

The  ECM  study  focuses  on  the  184  long  and  short  pods  and  the  131  three- 
band  pods.  Charts  and  sample  data  in  this  document  are  from  184  long 
pods. 


-2- 


This  Analysis  Focuses  on  ECM  Pods  (131,184) 
Critical  Components  of  EAF  Combat  Operations 


The  ECM  pod  is  a  single  pod  mounted  under  the  fuselage  or  wing  of  an 
aircraft.  The  pod  allows  for  electronic  scrambling  of  enemy  radar, 
improving  the  survivability  of  engaged  aircraft. 

As  of  October  1999,  the  U.S.  Air  Force  inventory  included  2300  aircraft 
configured  for  ECM  pods,  including  all  blocks  of  F-16A,  B,  C,  and  D  and 
all  A-10  models.  Although  there  are  several  variations  of  ECM  pods,  our 
analysis  focuses  on  AN  /  ALQ-1312  three-band,  AN  /  ALQ-184  long,  and 
AN  /ALQ-184  short  pods  because  they  represent  the  majority  of  the 
current  inventory,  and  a  significant  amount  of  data  associated  with  these 
pods  is  available  in  RAMPOD. 


2AN  /ALQ  signifies  airborne  countermeasure  equipment  that  serves  spedal  or 
combination  purposes.  There  were  originally  two  blocks  of  two-band  and  three-band 
131  pods.  The  Block  I  pods  were  retired  as  the  Block  II pods  became  available.  The 
remaining  two-band  pods  have  been  converted  to  three  bands.  The  184  long  pods,  the 
older  of  the  two  184  models,  are  three-band  technology,  while  the  short  pods  are  two- 
band  technology. 
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Analysis  of  RAMPOD  Web  Site  as  a 
Useful  Data  Source 


•  Overview  of  RAMPOD 

•  Analytic  Approach  Framin  g  the  Analysis 

•  Proposed  model  of  ECM  pod  ILM  process 

•  Proposed  data  elements  in  ECM  pod  study  supported 
by  RAMPOD 

-  Beddown 

•  Available  data 

•  Application 

•  Recommendations 

-  Pod  removals 

-  Process  queues 

-  Repair  process 

•  Summary 

rand  Project  air  force  i 


The  chart  shown  above  provides  an  overview  of  the  remainder  of  the 
document.  First,  we  give  an  overview  of  the  format  and  type  of  data 
contained  in  RAMPOD.  We  then  describe  the  analytic  approach  and 
methodology  of  RAND's  overarching  ACS  studies.  Next,  we  discuss  the 
proposed  model  architecture  of  the  ECM  ILM  process  and  how 
RAMPOD  data  supports  this  model.  Subsequently,  we  discuss  the 
specific  data  elements  in  the  ECM  pod  repair-process  model  that 
RAMPOD  supports,  how  the  data  can  be  used  in  the  analysis,  and 
potential  improvements  to  the  availability  and  display  of  the  data.  We 
discuss  the  beddown,  removal  rate,  and  repair-process  elements  in  great 
detail  and  offer  opportunities  for  improving  repair-shop  operational 
performance  monitoring  through  enhancements  to  the  RAMPOD  system. 
We  close  with  a  summary  of  our  recommendations  for  the  RAMPOD 
web  site  and  database. 
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RAMPOD  Is  a  System  That  Reports  Reliability, 
Availability,  and  Maintainability  Data  of  Pods 


.  Web-based 

•  Has  preprogrammed 
queries 


Systems 


Official  HJi.  tJwcranwnl  System  F»r  Anthomed  ttee  Oith.Do  N«|  DUruss  ClassiBeJ  Sensirire  Nioanal  Secnrife 
Iniormaiimi »f  Ciratrr  Smsitmrv  Tliaa  Tkat  I'd t  WMrb  lilts  Syslem  k  AuUm-urd |l*e  Of  Tkis  System  Cnsdnlrs 
Cinsenl  T«  Secuiitv  Testin'  And  Monimiin;.  lliuatlmiited  Use  CnuU  Result  in  Criminal  finserutinn. 


•  Flexibility  of  choices 
differs  from  query  to  query 

-  Level  of  detail:  MAJCOM, 
base,  pod,  serial  number 

-  Total  history  or  past 
year 

•  Displayed  in  text  format 

•  RAMPOD  team  is  a  helpful 
resource 


-  Frank  Hays 


https://www.rampod.robins.af.mil/ 
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OVERVIEW  OF  RAMPOD 

We  now  offer  a  brief  overview  of  the  RAMPOD  database  followed  by  an 
assessment  of  its  applicability  to  our  ECM  modeling  efforts.  The 
RAMPOD  web  site  is  located  at  https:  /  /  www.rampod.robins.af.mil. 
RAMPOD  is  a  reliability,  availability,  and  maintainability  logistics 
engineering  support  system  for  electronic  combat  pods  and  integrated 
systems.  RAMPOD  information  is  displayed  in  a  web-based  format  and 
includes  data  for  ECM  pods,  their  associated  support  equipment 
(limited),  LANTIRN  (being  integrated),  integrated  avionics  systems 
(limited),  and  air  combat  training  systems  (limited).  RAMPOD  tracks 
more  than  1300  ECM  pods  at  50  bases  worldwide  and  has 
preprogrammed  queries  that  can  be  changed  within  certain  limits  to  suit 
the  user.  Much  of  the  data  can  be  summarized  by  major  command 
(MAJCOM);  base;  mission,  design,  and  series  (MDS);  or  pod  serial 
number.  After  the  query  is  selected,  the  data  is  displayed  in  a  text 
format.  The  RAMPOD  team  is  also  a  helpful  resource.  Frank  Hays  has 
aided  us  considerably  in  acquiring  some  of  the  RAMPOD  data  pertinent 
to  our  study  in  a  more  complete  and  flexible  format  for  our  analytical 
needs.  Some  of  this  data  will  be  shown  later  in  the  report. 
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Data  Query  Structure 


RAMPOD's  ECM  pod  data  is  organized  into  three  sections: 
maintenance,  statistics,  and  inventory  and  status.  The  maintenance 
section  has  both  component  and  pod  data.  Component  data  available 
includes  most  common  part  removals,  failure  event  data,  and  part 
removal  detail  referenced  to  specific  pods.  Pod  data  for  each  pod  serial 
number  includes  aircraft  and  bench  hours,  failures,  critical  failures,  and 
times  between  maintenance  and  failures  as  well  as  individual 
maintenance  records.  The  statistics  section  has  both  system  and 
component  metrics.  System  metrics  for  each  MDS  include  mean  time 
between  failure  (MTBF),  mean  time  between  critical  failure,  mean  time 
between  repair,  mean  time  between  maintenance,  mean  time  to  repair 
(MTTR),  mean  turnaround  time  (MTAT)  (for  184  pods  only),  and 
operating  hours,  aggregated  by  base  or  MDS.  Component  metrics 
include,  for  individual  parts  on  184  pods  only,  operating  hours,  MTBF, 
mean  time  between  critical  failure,  and  mean  time  between  demand.  The 
inventory  and  status  section  has  pod  location  and  inventory  organized 
by  either  the  pod-owning  base  or  the  actual  operating  location,  along 
with  mission-capable  rates  as  designated  by  fully  mission  capable  (FMC), 
awaiting  maintenance  (AWM),  awaiting  parts  (AWP),  work  in  process, 
and  condemned.  RAMPOD  reports  daily  the  most  current  inventory  and 
status  updates  from  units,  although  not  all  bases  are  shown.  It  has  also 
been  reporting  monthly  mission-capable  rates  since  January  2000.  The 
inventory  and  status  section  shows,  for  each  base,  how  many  pods  are  in 
each  status. 
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184  Long  Repair  Data  Increased  with  Inventory 
Buildup  but  Has  Declined  over  Last  Five  Years 


184  long  pod  repairs  per  year 


-t  300 


3  Unscheduled 
i  Scheduled 


1987  1988  1989  1990  1991  1992  1993  1994  1995  1996  1997  1998  1999 
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In  its  current  form,  RAMPOD  cannot  generate  graphical  output;  thus,  the 
next  three  overview  charts  were  developed  using  historical  data  pulled 
from  the  underlying  data  warehouse  supporting  RAMPOD.  As  the  chart 
above  shows,  there  has  been  a  significant  increase  in  the  amount  of  data 
reported  to  and  displayed  in  RAMPOD.  This  chart  shows  the  changes  in 
the  number  and  type  of  repair  data  entries  in  RAMPOD  over  the  last  13 
years  for  184  long  pods.  Each  repair  type  is  broken  out  for  each  column. 
The  actual  number  of  pods  in  inventory  each  year  is  represented  by  the 
dotted  line  overlaid  on  the  columns.  It  should  be  noted  that  this  data 
was  acquired  in  November  of  1999,  so  the  last  few  months  of  1999  data 
are  absent,  making  1999  data  incomplete.  This  is  also  the  case  with 
several  of  the  charts  that  follow. 

The  RAMPOD  data  warehouse  contains  much  data  that  cannot  be 
readily  accessed  through  the  current  web-based  tool;  thus,  unless 
otherwise  specified,  data  for  charts  and  discussion  was  extracted  by 
Frank  Hays  upon  special  request. 
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131  Three-Band  Repair  Data  Increased  with 
Inventory  Buildup 
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aPMI  =  periodic  maintenance  inspection. 
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This  chart  displays  the  same  type  of  data  as  in  the  previous  slide,  but  for 
the  131  three-band  pods.  The  rise  in  pod  inventory  and  maintenance 
entries  is  due  to  the  modification  of  two-band  pods  to  three-band  pods 
beginning  in  the  mid-1990s.  The  steady  rise  in  pods  is  accompanied  by  a 
similar  rise  in  maintenance  events,  suggesting  a  fairly  steady  ratio  of 
repairs  per  pod  during  this  time  period.  This  slide,  like  the  previous 
one,  shows  the  kind  of  data  that  RAMPOD  contains.  Again,  the  1999 
repair  data  is  incomplete. 
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Many  Maintenance  Records  Had  Missing  Data 


Maintenance  records  with  missing  field,  RAMPOD  1997-4999 


Hours  Hours 

ALQ-184  ALQ-131 
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One  of  the  issues  we  found  with  the  RAMPOD  data  centered  on  quality 
— in  this  case  the  number  of  entries  that  are  missing  various  data 
elements.3  The  chart  above  shows  the  proportion  of  missing  data  in  two 
of  the  fields  in  the  maintenance  records — operating  hours  and  bench 
hours — between  1997  and  1999  for  the  184  and  131  pods.  These  two 
fields  were  chosen  because  of  their  relevance  to  the  larger  ECM  study. 
The  lower  part  of  each  column  represents  the  number  of  records  that  had 
data  in  the  field  for  the  maintenance  record;  the  top  section  shows  the 
number  of  records  that  had  no  data  in  that  subject  field. 

Although  the  percentages  of  missing  data  are  significant,  estimates  using 
the  remaining  data  can  still  be  used.4  The  issue  is  the  desired  precision 
of  the  statistics.  For  Air  Force-wide  or  MAJCOM-wide  measurements, 
depending  on  the  time  period  measured,  an  adequate  sample  should  be 
easy  to  acquire.  For  many  of  the  bases,  however,  one  year  of  data  would 
probably  not  yield  a  large  enough  sample  size  to  ensure  confidence  that 
the  estimate  was  representative  of  their  pods'  performance.  Several 
bases  were  missing  50  percent  of  their  operating-hour  or  bench-hour 

3For  a  further  discussion  on  data  quality  problems  and  their  impact  on  analyses,  see 
Galway  and  Hanks  (1996). 

4This  assumes  that  the  missing  portion  of  the  data  is  similar  to  the  data  available  and 
bases  statistical  confidence  only  on  real  entries. 
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data.  Since  a  lower  percentage  of  total  records  captured  would  decrease 
the  likelihood  that  the  estimated  mean  (or  other  statistics)  matched  the 
actual  mean,5  this  would  leave  very  few  records  for  smaller  bases,  and 
many  base-to-base  comparisons  would  therefore  be  suspect.  Also,  if 
smaller  time  periods — months  or  quarters — were  used  for  these 
statistics,  only  bases  with  many  pods  and/or  high  pod  usage  would 
have  adequate  sample  sizes  for  analysis. 

Although  several  data  quality  and  quantity  improvements  may  be 
possible,  a  more  comprehensive  analysis  (e.g.,  one  that  examined  several 
data  elements  in  several  sections)  would  better  quantify  RAMPOD's  data 
quality  before  implementation.  Presumably  the  missing-data  problem 
originates  during  maintenance,  when  data  is  recorded  from  the  pod.  A 
first  step  could  be  to  give  feedback  to  the  units  informing  them  how 
much  of  their  data  is  missing.  This  could  come  in  the  form  of  a  simple 
monthly  report  stating  the  total  number  of  maintenance  records  reported 
in  the  previous  month  and  the  percentage  of  missing  data  in  each  field. 

It  could  also  include  some  aggregate  numbers  for  other  bases  or  the 
overall  force  for  comparison.  Simply  showing  the  units  how  complete 
their  data  is  (especially  in  comparison  to  other  units)  could  draw  enough 
attention  to  the  issue  to  encourage  better  recording  and  reporting.  With 
this  regular  feedback,  base  commanders  could  know  how  well  their  data 
is  being  reported  to  RAMPOD  and  therefore  determine  how  relevant  it  is 
for  self -measurement. 

Another  way  to  give  units  this  kind  of  feedback  is  to  use  only  data  from 
bases  that  had  an  adequate  sample  and  notify  those  bases  that  were  left 
out  for  this  reason.  This  could  additionally  highlight  the  importance  of 
reporting  good  data. 

A  different  kind  of  solution  could  He  in  making  RAMPOD's  data  display 
more  user-friendly  and  more  applicable  to  individual  units.  Graphical 
displays — discussed  in  greater  detail  later  in  this  report — can  be  concise 
and  easy  to  understand;  thus,  they  could  allow  for  more  widespread  and 
regular  use  of  the  RAMPOD  web  tool  among  individual  units.  The  more 
the  units  use  the  web  tool  for  performance  reporting,  the  more  likely 
they  are  to  encourage  accurate  recording  and  reporting  of  data.  It  is 
possible  that  the  more  people  use  RAMPOD,  the  more  attention  the  data 
would  be  given. 

This  overview  highlights  some  opportunities  for  operational  and 
strategic  analyses  possible  with  the  RAMPOD  system — although  not  in 
its  current  format.  We  detail  these  opportunities  in  the  charts  that 
follow. 


5For  a  further  discussion  of  statistical  sampling  and  estimation,  see  Smith  (1988). 
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RAM  POD  Data  Warehouse  Offers  Several 
Performance  Measurement  Opportunities 


•  Tracking  pod  performance  trends 

-  Permits  detection  of  systemic  or  local  problems 

•  Detailed  breakout  of  data  at  different  levels 

-  Allows  comparison  of  performance  and  benchmarking  of 
processes  between  bases  or  MAJCOMs 

Missing  data  may  adversely  affect  performance  measures 
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The  data  already  reported  to  and  stored  in  RAMPOD  has  great  potential 
for  supporting  operational  and  strategic  analyses  of  ECM  pods  as  well  as 
the  other  weapon  systems  being  integrated  into  the  database.  These 
opportunities  will  be  expanded  on  throughout  this  report.  The  first  is  to 
track  and  display  trends  for  removals,  repair  times,  and  in-shop  queue 
using  data  already  captured  by  RAMPOD.  This  could  permit  the 
detection  of  systemic  or  local  problems  at  an  operational  level  and  allow 
for  more  comprehensive  strategic  planning.  Next,  the  level  of  detail  at 
which  data  is  reported  can  enhance  analysis.  Aggregating  data  at 
different  levels  allows  for  comparison  of  performance  and  benchmarking 
between  bases  or  MAJCOMs.  Finally,  as  noted  earlier,  the  amount  of 
missing  data  can  adversely  affect  analysis,  so  improving  reporting  to  the 
system  could  increase  confidence  in  analyses  supported  by  RAMPOD. 
We  expand  on  these  opportunities  throughout  this  report. 
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Assessment  Models 
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ANALYTIC  APPROACH 

All  of  our  EAF  support  posture  evaluations  use  the  employment-driven 
analysis  approach  shown  in  the  chart  above.  The  first  step,  shown  on  the 
left,  uses  force-employment  models  to  identify  the  force  packages 
necessary  to  successfully  accomplish  anticipated  missions  (e.g.,  the 
numbers  of  each  aircraft  type  and  their  flying  requirements  for  each 
scenario).  In  this  case,  the  information  is  used  to  estimate  the  demand 
for  ECM  pod,s  which,  together  with  the  support  concept  being  modeled, 
drives  the  requirements  for  maintenance  equipment,  maintenance 
personnel,  spare  parts,  and  transportation  resources,  as  represented  in 
the  center  of  this  chart.  We  then  determine  the  costs  of  each  alternative 
and  evaluate  them  against  the  operational  requirements,  and  the  results 
obtained  forecast  the  effects  of  potential  support  options.  If  the 
alternatives  do  not  meet  operational  needs,  the  method  can  be  used  to 
evaluate  possible  revisions  of  operational  objectives  or  to  develop 
alternative  support  practices  or  technologies  to  lift  constraints. 

The  alternative  support  structure  designs  are  defined  by  peacetime  and 
wartime  locations  of  ECM  pod  aircraft  intermediate  maintenance  assets. 
These  locations  drive  the  quantities  of  four  resources:  intermediate  test 
stands  and  fixtures,  personnel,  spare  parts,  and  transportation  assets. 

We  extend  this  approach  to  assess  possible  investment  options  and  their 
effect  on  support  system  capabilities  and  hence  resource  requirements. 
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The  ECM  pod  analysis  begins  with  the  employment-driven  resource 
models  to  determine  the  minimum  resource  levels  that  enable  each 
support  structure  to  meet  the  two-MTW  (major  theater  war)  scenario's 
operating  demands.  After  determining  the  composition  of  each 
alternative  structure,  the  analysis  evaluates  it  against  the  goals  defined 
by  the  EAF  objectives  as  well  as  peacetime  operations.  Through  an 
iterative  process,  we  develop  a  solution  space  encompassing  the  various 
scenarios  and  support  options. 

The  highlighted  areas  show  where  RAMPOD  contains  data  that  can 
support  this  modeling  framework.  It  contains  data  on  pod  failure  rates 
and  repair-shop  processes,  both  of  which  are  drivers  of  the  support 
requirements.  We  examine  the  applicability  of  RAMPOD  to  these  model 
elements  in  the  following  section. 
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RAMPOD  Data  Can  Support  Modeling  of  Beddown, 
Pod  Failure  Rates,  and  Repair-Shop  Processes 
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PROPOSED  MODEL 

The  chart  shown  above  describes  the  basic  elements  of  our  proposed 
analysis  model  as  applied  to  ECM  pod  employment  and  support 
structures.  It  also  highlights  elements  of  the  process  that  RAMPOD  data 
supports. 

The  loop  on  the  left  side  of  the  chart  describes  the  system  demand. 
Starting  with  a  given  beddown  and  a  specific  employment  program,  we 
can  predict  the  number  of  pods  removed  from  the  aircraft.  We  will 
specifically  model  removals  to  the  back  shop,  or  intermediate  level,  not 
those  at  the  flight  line.  Whereas  RAMPOD  offers  limited  removal-rate 
data  due  to  minimal  peacetime  ECM  pod  operational  requirements, 
RAND  will  develop  more  robust  failure-rate  relationships  using 
RAMPOD  data  in  concert  with  data  collected  during  the  Air  War  over 
Serbia.  Once  removed,  the  pods  must  be  transported  to  the  back  shop, 
which  may  be  on  base  or  offsite.  In  the  shop,  the  pods  await  repair  until 
capacity  and  /  or  parts  are  available.  The  in-shop  queue  and  repair  times 
can  be  based  on  actual  data  from  RAMPOD,  such  as  the  elapsed  time 
indicator  (ETI)  clock  times  and  powered-off  repair.  After  repair,  the 
pods  must  be  transported  back  to  the  flight  line.  In  the  on-base  repair 
option,  transportation  is  usually  by  trailer,  whereas  in  the  consolidated 
repair  approach,  transportation  may  be  by  air  or  truck.  To  summarize, 
the  left  process  loop  on  this  chart  generates  a  certain  time  demand  on  the 
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system.  Now  let  us  consider  the  supply  side  of  the  model.  We  included 
three  major  elements  of  supply:  stockage  (the  number  of  pods  or  line- 
replaceable  units  [LRUs]  available),  the  number  of  test  sets  needed,  and 
the  number  of  people  required  given  various  work  schedules, 
productivity  rates,  and  logistics  structures.  Combining  these  elements, 
we  can  assess  resource  allocation  and  availability.  These  elements  will  be 
outputs  of  our  models  and  thus  do  not  require  data  from  RAMPOD. 

In  each  scenario,  the  goal  will  always  be  to  have  supply  greater  than  or 
equal  to  demand.  To  summarize,  RAMPOD  can  help  us  develop 
bed  downs  as  well  as  removal-rate  relationships,  in-shop  queuing 
models,  and  repair-time  estimates. 
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Process  Road  Map 


PROPOSED  DATA  ELEMENTS 

The  chart  above  will  serve  as  the  process  road  map  for  the  remainder  of 
the  document,  with  subsequent  sections  highlighted  on  the  chart.  Next, 
we  discuss  each  element  of  the  ILM  process  for  which  RAMPOD  has 
data.  In  each  section,  we  discuss  what  data  is  available,  how  this  data 
can  be  used,  and  potential  improvements  to  the  current  reporting  system. 
As  indicated  earlier,  the  RAMPOD  data  warehouse  contains  much  data 
that  cannot  be  readily  accessed  through  the  current  web-based  tool. 


RAM  POD  Beddown  Data  Applicable  to  Analysis 


Current 

Proposed 

0  Pod  inventory 

0  Pod  status 

0  Support  equipment  inventory 

□  Integrated  beddown,  including 
pods,  support  equipment 
aircraft,  and  personnel 

0  Support  equipment  status 

□  Personnel  experience 

□  MTBF 

□  Time  between  failure 

□  Separate  transmitter  times 

□  MTAT 

□  Awaiting  maintenance 

□  Awaiting  parts 

□  MTTR 

□  Time  to  repair 

□  Separate  repair  types 

□  Single  number 

□  Distributions,  percentiles,  trends 

□  Text 

□  Graphical 
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Beddown 

RAMPOD  includes  two  elements  of  the  beddown  we  propose  to  use  for 
the  ECM  study.  It  has  up-to-date  data  on  the  location,  inventory 
position,  and  operational  status  of  ECM  pods  (131  and  184)  as  well  as  the 
number  and  location  of  the  test  equipment  used  to  repair  these  pods 
(233D  and  256).  This  data  can  be  summarized  by  MAJCOM  or  base  for 
each  MDS  and  gives  a  recent  snapshot  of  the  availability  of  pods  and  test 
equipment.  Next,  we  show  how  our  analytic  models  can  use  this  data. 
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RAMPOD  Can  Be  Used  to  Develop  Pod  and 
Support  Equipment  Beddown 


The  objective  of  the  ECM  analysis  is  to  determine  the  costs  and 
operational  benefits  of  alternative  ECM  pod  maintenance  structures  that 
can  satisfy  the  entire  spectrum  of  operational  requirements,  including  the 
Defense  Planning  Guide's  two-MTW  scenario,  a  one-MTW  scenario, 
small-scale  Air  Expeditionary  Forces  (AEFs),  and  boiling  peacetime 
operations.  When  modeling  an  employment  scenario,  we  must  begin 
with  a  corresponding  deployment  plan  and  an  initial  force  beddown. 

There  are  four  components  to  the  beddown  necessary  for  our  models. 

We  use  the  location  and  inventory  of  ECM  pods  (131  and  184),  test 
equipment  (233D  and  256),  and  aircraft  (F-16  and  A-10).  We  also  use  the 
location  and  skill  level  of  the  personnel  assigned  to  ECM  pods.  Although 
RAMPOD  has  pod  and  test  equipment  inventory,  aircraft  and  personnel 
data  need  to  be  attained  from  other  sources  such  as  Air  Combat 
Command  (ACC). 
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Potential  Improvements  and  Benefits 


Current 

Proposed 

□  p 

□  P 

□  s 

Benefits 

0  Integrated  beddown,  including 
pods,  support  equipment, 
aircraft,  and  personnel 

□  s 

•  Offers  comprehensive  snapshot 
of  force  availability 

0  Personnel  experience 

□  M 

•  Facilitates  data  collection  efforts 

□  Time  between  failure 

□  M 

•  Allows  for  better  planning  of 
manning  requirements 

□  Separate  transmitter  times 

_ _ _ P 

□  Awaiting  maintenance 

□  Awaiting  parts 

□  MTTR 

□  Time  to  repair 

□  Separate  repair  types 

□  Single  number 

□  Distributions,  percentiles,  trends 

□  Text 

□  Graphical 
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It  would  be  beneficial  to  the  Air  Force  to  have  an  integrated  beddown  of 
ECM  pods,  test  sets,  aircraft,  and  personnel  at  a  greater  level  of  detail 
(e.g.,  MAJCOM,  base).  Since  the  units  are  already  responsible  for 
reporting  pod  and  support  equipment  inventory  and  aircraft  FMC  rates, 
the  addition  of  personnel  information  would  add  little  time  to  the 
reporting  process.  Since  aircraft  data  is  not  currently  reported  into 
RAMPOD  but  is  instead  aggregated  at  the  MAJCOM  level,  one  option  is 
to  link  this  information  with  the  RAMPOD  database.  This  integrated 
approach  would  yield  a  comprehensive  picture  of  pod-appropriate  force 
availability  at  a  given  time,  thereby  allowing  for  easier  analysis  for 
deployment  scenarios. 

Another  possibility  is  to  supply  experience-level  data  of  personnel 
assigned  to  pods.  This  could  allow  for  better  planning  of  manning  for 
shops.  Again,  this  type  of  data  is  already  collected,  so  linking  existing 
databases  to  RAMPOD  may  offer  additional  enhancements  at  a  relatively 
low  cost  to  the  Air  Force.  Changes  like  these  broaden  RAMPOD's 
applicability  as  an  information  source,  which  could  increase  its  visibility 
and  analytical  value  throughout  the  Air  Force. 

Whereas  these  enhancements  primarily  offer  improvements  supporting 
strategic  analyses,  the  following  discussion  focuses  on  opportunities  for 
improving  daily  operational-performance  monitoring. 


-19- 


Process  Road  Map 


Pod  Removals 

Next,  we  discuss  the  flight-line  and  back-shop  pod  removal  data  in 
RAMPOD. 
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RAMPOD  ECM  Pod  Removal  Data  Applicable 

to  Analysis 


Proposed 


Current 


□  Pod  inventory 

□  Pod  status 

□  Support  equipment  inventory 

□  Support  equipment  status 

0  MTBF 

□  MTAT 

□  MTTR 

0  Single  number 
0  Text 


□  Integrated  beddown,  including 
pods,  support  equipment 
aircraft,  and  personnel 

□  Personnel  experience 

0  Time  between  failure 
0  Separate  transmitter  times 

□  Awaiting  maintenance 

□  Awaiting  parts 

□  Time  to  repair 

□  Separate  repair  types 


0  Distributions,  percentiles,  trends 
0  Graphical _ 
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RAMPOD  has  detailed  data  on  pod  removals  from  Air  Force  maintenance 
records.  The  available  data  in  RAMPOD  includes  the  number  of 
removals  per  time  period  and  a  computed  time  between  failures.  The 
charts  that  follow  look  at  some  of  this  data  from  different  perspectives 
and  at  different  levels  of  detail.  Since  RAMPOD  does  not  have  graphical 
capabilities,  the  charts  we  show  offer  another  enhancement 
opportunity — graphical  displays.  We  will  show  displayed  RAMPOD 
data  of  MTBF,  pod  removals  per  month,  distribution  of  time  between 
failure,  pod  removal-rate  percentiles,  and  pod  removals  over  time.  With 
this  data  one  can  identify  trends  in  pod  failures,  possibly  signaling 
systemic  or  base-level  problems.  One  can  also  see  the  entire  distribution 
of  pod  time  between  failures  instead  of  just  the  average,  or  MTBF. 
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RAMPOD  s  Present  Data  Shows  MTBF  Only 


184  long-pod  MTBF,  March  1999-February  2000 
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The  chart  above  shows  data  as  taken  directly  from,  but  not  charted  in, 
RAMPOD.  In  RAMPOD,  the  only  two  time  periods  for  which  data  can 
be  displayed  are  the  past  year  and  the  entire  recorded  history.  This  chart 
shows  MTBF  data  for  the  184  long  pod  for  the  past  year — March  1999 
through  February  2000.  MTBF  is  calculated  by  dividing  total  pod 
operating  hours  for  each  base  by  pod  failures  for  the  same  time  period. 
Several  bases  had  few  pod  operating  hours  during  the  time  period 
captured  by  this  query,  so  for  those  bases  this  statistic  could  reflect  the 
operation  of  only  one  or  two  pods.  An  average  value  based  on  so  few 
pods  could  be  misleading  if  the  pod(s)  failed  very  quickly  or  very  slowly. 
Operating  hours  are  therefore  shown  in  the  chart  to  qualify  the  MTBF 
from  each  base.6 

One  year  is  the  smallest  increment  available,  but  data  is  not  displayed  for 
all  bases.  Although  there  were  entries  for  14  bases,  this  chart  shows  only 
eight  of  them.  The  rest  were  missing  the  annual  statistic.  It  is  uncertain 
whether  missing  data  is  due  to  a  lack  of  failures  at  the  respective  bases  or 
if  data  was  not  entered.  Although  data  charted  in  this  manner  can 


6RAMPOD  also  shows  operating  hours  with  statistics  such  as  MTBF  and  MTTR  when 
available. 
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compare  pod  performance  from  base  to  base,  a  year  may  not  be  the  best 
time  period  across  which  to  aggregate  this  statistic.  Examples  of  data 
displayed  differently  that  reveal  more  about  pod  behavior  follow.7 

"For  a  comprehensive  treatment  of  statistical  chart  design,  see  Schmid  (1983). 


Distributions  Reveal  More  Than  Averages 


184  long-pod  time  between  failure,  all  bases,  January  1997-dune  1999 
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The  MTBF  metric  described  in  the  previous  chart  is  an  average  and  can 
mask  the  actual  distribution  of  removal  rates  among  the  population  of 
pods.  This  chart  shows  the  distribution  of  all  times  between  failures  for 
184  long  and  short  pods.  Using  RAMPOD  data  between  1997  and  1999, 
we  captured  the  operating  hours  of  each  pod  every  time  it  was  removed 
for  a  failure.  The  curve  on  the  chart  shows  the  number  of  times  a  pod 
was  removed  for  a  given  range  of  operating  hours  on  its  clock  when 
removed  for  a  failure.  One  can  see  the  large  range  of  failure  times,  which 
would  be  hidden  by  the  average  (shown  as  the  vertical  line).  A 
significant  number  of  pods  were  removed  after  zero  operating  hours  on 
the  aircraft,  signaling  either  an  immediate  failure  or  missing  data. 
Although  we  can  use  this  kind  of  data  to  check  one  of  the  inputs  of  the 
ECM  pod  support  options  model,  simple  time  between  failure  will  not 
be  the  only  input  for  determining  pod  removal  rates  in  the  ECM  pod 
study.  We  will  investigate  variables  such  as  sortie  frequency  and 
duration,  pod  shelf  life,  pod  age,  time  in  conflict,  and  pod  transmit  time 
to  determine  what  drives  pod  removals  and  how  it  is  done.  The 
additional  data  required  for  our  analysis  comes  from  data  collected 
during  the  Air  War  over  Serbia  as  well  as  from  elements  from  the 
RAMPOD  data  warehouse. 
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Distributions  Reveal  More  Than  Averages 


The  chart  above  shows  the  same  data  as  the  previous  one  did,  but  in  a 
more  concise,  percentile  ranking  format.  Here,  the  bottom  section  of  the 
column  shows  the  time  between  failure  value  below  which  half  of  all 
pods  failed.  In  this  instance,  half  of  the  pods  failed  after  44  or  fewer 
operating  hours.  The  second  section  shows  the  time  below  which  75 
percent  of  the  pods  failed  and  the  top  section  the  time  below  which  95 
percent  of  the  pods  failed.  The  MTBF  is  shown  as  a  square  dot  within 
the  75  percent  bar.  Displaying  the  data  in  this  format  reveals  how  pods 
in  the  higher  range  skewed  the  average  upward,  well  above  the  median, 
or  middle,  value.  These  distributions  allow  for  better  assessment  of 
current  operations  variability  as  well  as  for  the  computation  of  the 
resource  requirements  needed  to  support  these  pods.  This  particular 
display —  which,  unlike  the  distribution  shown  in  the  previous  chart, 
uses  a  percentile — can  then  be  used  to  analyze  trends,  as  shown  in  the 
next  chart.  Again,  RAND's  ECM  pod  study  will  evaluate  the  validity  of 
using  MTBF  exclusively  as  a  predictor  of  pod  failures. 
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As  mentioned  earlier,  a  benefit  of  showing  data  over  time  is  to  signal 
possible  developing  problems  with  the  pods.  The  chart  above  shows  the 
percentiles  of  removal  rates  for  184  long  pods  between  January  1997  and 
September  1999.  This  takes  a  similar  look  at  the  same  data  from  the 
previous  two  charts  but  examines  smaller  time  periods  to  see  how  the 
population  of  pod  failures  changes  over  time. 

Displaying  data  in  this  format  could  be  helpful  for  daily  operations.  The 
percentiles  show  the  variation8  of  the  removal  rates,  as  discussed  in  the 
previous  chart,  depicting  more  than  an  average  value  does.  The  more 
this  variation  occurs  in  pod  performance,  the  more  that  unnecessary 
failures  may  occur.  Tracking  pod  failure  data  in  the  above  format  allows 
for  the  identification  of  special  causes  of  variation  that  result  in  a  shift  in 


8Variation  occurs  in  any  process.  This  variation  is  due  either  to  random  causes  (the 
cum ulative  effect  of  many  small,  unavoidable  causes)  or  to  special  causes  (defects  that 
are  not  part  of  the  chance  causes)  and  can  usually  be  identified  and  corrected.  A 
process  that  exhibits  many  special  or  assignable  causes  of  variation  is  considered  out  of 
control.  In  a  manufacturing  process,  some  random  variation  is  normal  and  tolerable. 
Special  causes  of  variation  result  in  unnecessary  defects  that  in  turn  cause  poor  product 
performance  resulting  from  nonconformity  and  failures.  Statistical  process  control,  a 
methodology  for  tracking  and  eliminating  process  variations,  uses  many  statistical  and 
charting  tools,  including  some  like  the  one  above.  For  more  detail  on  statistical  process 
control,  see  Montgomery  (1985)  or  Pyzdek  (1989). 
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the  time  between  failure  and/ or  the  variation  of  these  times.  Finding 
and  eliminating  these  special  causes  can  increase  the  time  between 
failure  and  decrease  the  variability  in  the  process.  One  of  the  benefits  of 
this  decreased  variability  is  an  increased  ability  to  accurately  predict  the 
outcome  of  the  process,  in  this  case  the  time  between  failure.  This 
predictive  ability  aids  resource  planning.  Charting  data  periodically 
allows  one  to  track  changing  times  between  failure  signaling  problems 
and  to  observe  changes  in  the  system  after  a  process  improvement  or 
technology  change.  The  lower  times  between  failure  in  the  last  two 
quarters  of  the  chart  indicate  performance  degradation.  Indicators  like 
this  do  not  show  what  the  problem  is,  only  that  more  analysis  is  needed. 
One  should  break  out  the  data  to  look  at  specific  bases  or  MAJCOMs  to 
see  if  the  problem  can  be  pinpointed.  The  more  user-friendly  the  data  in 
RAMPOD,  the  easier  it  will  be  for  more  people  to  conduct  this  type  of 
investigative  analysis. 
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Potential  improvements  and  Benefits 
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RAMPOD  currently  shows  a  snapshot  of  MTBF  and  other  failure  data  for 
each  MDS  or  base  for  the  total  history  and  the  past  year.  It  does  not, 
however,  display  particular  past  months  or  years.  The  first 
recommendation  for  failure  data  is  thus  to  track  and  display  periodic 
data  in  chart  form.  Showing  data  over  a  period  of  time  can  reveal 
incremental  changes  that  could  otherwise  go  unnoticed.  The  time  period 
used  for  this  statistic  should  be  chosen  carefully.  As  already  shown, 
some  bases  did  not  have  many  operating  hours  even  over  a  year.  Some 
bases  may  fly  pods  often  enough  to  merit  a  smaller  time  period  for  data 
display,  but  the  decision  should  be  made  on  sample  size  in  order  to  have 
statistically  sound  data  displayed.  Also,  graphical  representation  of  data 
is  much  easier  to  understand,  allowing  for  faster  and  more  accurate 
performance  analysis  that  could  ultimately  encourage  more  widespread 
use  of  RAMPOD  among  individual  units.  As  emits  use  RAMPOD  more 
often,  they  may  be  motivated  to  report  their  data  more  accurately  as  well. 
Observing  similar  trends  at  other  bases  or  MAJCOMs  could  signal  an 
overall  problem,  while  different  trends  elsewhere  could  signal  an 
important  difference  in  pod  performance.  Also,  data  collected  at  the 
beginning  of  the  program  was  less  consistent  and  could  compromise  the 
accuracy  of  a  total  history  statistic.  Similarly,  the  use  of  smaller  time 
periods,  where  appropriate,  can  reveal  changes  that  would  be  masked  by 
an  annual  average. 
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Another  suggestion  is  to  show  the  percentiles  and  /  or  distributions  of 
these  failure  times  for  trend  and  aggregate  analysis.  This  display  of  data 
gives  a  clearer  picture  of  how  the  population  of  pods  performed  at  the 
base,  MAJCOM,  or  force  level. 

Another  possibility  is  to  show  different  pod  clock  times  for  operating 
hours.  The  RAMPOD  data  warehouse  already  stores  all  pod  clock  times 
as  reported  from  maintenance  records  but  displays  only  operating 
hours — i.e.,  the  number  of  hours  the  pod  was  "on"  but  was  not 
necessarily  transmitting.  The  actual  transmit  time(s)  of  the  pods  could 
have  a  greater  effect  in  causing  failures  than  standby  time.  One  of  the 
goals  of  the  RAND  analysis  is  to  describe  as  accurately  as  possible  what 
drives  the  pod  removal  rates.  Having  all  of  the  operating  data  for  each 
pod  can  help  attribute  pod  failures  to  specific  operations  the  pods 
perform. 
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Process  Road  Map 


Repair  Process  Queues 

Next,  we  discuss  the  queue  time  before  repairs.  This  queue  is  composed 
of  time  AWM  (before  a  pod  can  be  diagnosed  or  serviced)  and  time  AWP 
(after  diagnosis  while  a  pod  awaits  necessary  repair  parts). 
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Awaiting  Parts  and  Maintenance  Data  Applicable 
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Currently,  one  can  retrieve  only  base-level  MTAT  for  a  repair  (the  total 
time  a  pod  spends  from  entry  to  exit  of  the  repair  shop)  and  MTTR  (the 
time  spent  servicing  the  pod).  By  subtracting  MTTR  from  MTAT,  we  can 
estimate  the  time  awaiting  maintenance  or  parts  and  shipment  out  of  the 
shop.  This  data  is  available  only  for  184  long  and  short  pods.  Although 
this  data  can  be  used  to  compare  wait  times  between  bases,  it  is  not 
specific  enough  to  point  to  a  particular  problem. 
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Mean  Turnaround  Time  and  Awaiting  Parts  or 
Maintenance  Data  Is  Limited 
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The  chart  above  shows  the  awaiting  parts  and  maintenance  (AWP/M) 
time  and  MTTR  for  several  bases  for  184  long  pods.  Although  there  is 
variation  in  repair  times,  a  significant  proportion  and  variation  of  the 
MTAT  is  due  to  time  AWM  or  AWP.  Whether  it  is  due  to  AWM  or  AWP 
is  unclear  from  the  available  data.  As  described  earlier,  this  wait  time 
must  be  derived  from  two  other  summary  statistics  that  are  available  on 
the  RAMPOD  web  site. 

This  data  has  operational  significance.  It  is  apparent  from  this  chart  that 
the  MTAT  could  be  significantly  reduced  by  decreasing  wait  times. 
Within  this,  each  wait  time  is  due  to  a  different  problem.  Time  AWM 
may  be  an  issue  of  manning  the  repair  shop.  The  other  issue  is  time 
AWP.  This  may  indicate  a  supply  system  problem.  The  slower  the 
supply  system,  the  more  time  repairs  take,  affecting  support  for  the 
pods. 

Despite  this  significance,  the  data  displayed  in  RAMPOD  is  still 
ambiguous — whether  wait  is  due  to  parts  or  maintenance  is  unknown. 
The  next  chart  contains  several  suggestions  for  improving  the  display  of 
this  data  and  for  improving  its  importance  to  daily  operations.  More 
strategic  analyses  are  also  given. 
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Potential  Improvements  and  Benefits 


Current 


□ 
a  f 
Qi 

□  J 

□  n 


0 


Benefits 

.  Allows  for  more  complete 
analysis 

•  Differentiate  between  supply 
system  issue  and  repair  shop 
issue 

•  Identify  trends 

.  More  accurately  portray  behavior 
of  process _ p-/ 


□  MTTR 

0  Single  number 
0  Text 


Proposed 


□  Integrated  beddown,  including 
pods,  support  equipment, 
aircraft,  and  personnel 

□  Personnel  experience 

□  Time  between  failure 

□  Separate  transmitter  times 

0  Awaiting  maintenance 
0  Awaiting  parts 

□  Time  to  repair 

□  Separate  repair  types 

0  Distributions,  percentiles,  trends 
0  Graphical _ 


|  RAND  Project  AIR  FORCE 


One  important  suggestion  is  to  display  data  for  all  MDSs — 131  pods  in 
particular.  It  is  uncertain  why  this  data  is  currently  not  in  RAMPOD,  but 
MDS  data  is  necessary  for  a  more  complete  analysis.  It  would  also  be 
beneficial  to  show  AWM  and  AWP  separately.  Since  these  two  times  are 
reflective  of  two  different  issues,  separating  them  would  lend  added 
insight  to  both  processes.  The  data  to  support  both  of  these  suggestions 
is  already  in  the  RAMPOD  data  warehouse,  so  adding  its  display  should 
not  prove  difficult. 

This  data  could  be  shown  at  several  levels  of  detail,  such  as  MAJCOM 
and  base,  because  much  of  the  RAMPOD  data  is  already  displayed. 
Showing  monthly  numbers  could  help  identify  trends,  as  has  been 
addressed  previously.  In  this  case,  a  tune  period  as  small  as  a  month 
could  certainly  be  appropriate.  The  performance  of  any  repair-shop 
process  is  essentially  independent  of  the  number  of  hours  a  base's  pods 
fly  in  a  given  time  period.  This  type  of  display  could  be  more  applicable 
to  the  shop  queue  and  actual  repair  process  than  to  MTBF,  since  the  first 
two  variables  should  always  have  adequate  data  points.  This  data 
should  probably  be  displayed  in  chart  form,  as  was  suggested  for 
removal  data. 

Showing  percentiles  and/or  distributions  of  wait  times  would  reflect  the 
range  of  times  and  would  therefore  contribute  to  a  better  understanding 
of  repair-system  performance. 
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Continuing  to  display  guard  and  active  data  separately  for  purposes  of 
comparison  opens  up  an  opportunity  to  observe  and  leverage  different 
processes.  Benchmarking  between  the  two  forces  could  allow  for 
improvements  using  solutions  already  being  implemented. 


Process  Road  Map 


Pod  Repairs 

Next,  we  discuss  the  repair  process  data. 


-35- 


ECM  Pod  Repair  Data  Applicable  to  Analysis 
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RAMPOD  has  comprehensive  maintenance  records  with  repair  type  and 
bench  time9  as  well  as  a  detailed  maintenance  history  for  each  pod. 
Because  the  available  data  is  either  highly  detailed  information  about 
individual  pods  or  parts  or  aggregated  across  an  entire  base,  we 
requested  additional  data  from  the  RAMPOD  office  to  allow  for  a  more 
complete  analysis.  We  have  charted  the  overall  distribution  of  repair 
times,  comparison  between  guard  and  active  forces,  the  number  of 
maintenance  events  per  month  over  several  years,  and  the  distribution  of 
repair  times  over  several  years.  With  this  data  one  can  identify  trends  in 
repair  times,  observe  a  range  of  bench  times,  and  contrast  times  for 
different  repairs.  Trend  analysis  could  be  used  to  predict  performance 
and  requirements,  while  repair  types  and  distributions  of  repair  times 
could  be  used  to  more  accurately  represent  the  ILM  repair-shop  process. 


“Bench  time  is  the  time  a  pod  is  actually  being  serviced  in  the  shop,  which  excludes 
AWP  or  AWM  time. 
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RAMPOD  s  Present  Display  of  Repair  Times  Is 
Limited  in  Application 


184  long-pod  MTTR  base  averages,  March  1999  —  Febuary  2000 
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The  chart  above  shows  MTTR  data  for  the  184  long  pod  for  the  past  year 
for  several  bases  taken  directly  from  the  RAMPOD  web  site  (the  chart 
was  not  generated  in  RAMPOD).  Although  data  charted  in  this  manner 
can  compare  pod  behavior  from  base  to  base,  it  is  still  aggregated  over 
many  pods  over  a  relatively  long  period  of  time,  thus  masking  both  the 
variation  in  the  process  and  changes  in  performance  over  smaller  time 
periods.  Understanding  variation  is  important  when  modeling  the 
process.  Gradual  changes  in  performance  are  important  for  strategic 
planning  as  well  as  for  operational  monitoring. 
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Distributions  Reveal  More  Than  Averages 


184  long-pod  bench  times,  January  1997 — Jue  1999 
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The  chart  above  shows  the  distribution  of  all  repair  times  for  184  pods, 
broken  out  by  repair  type.  One  can  see  the  differences  between  each 
type  of  repair  as  well  as  the  wide  variation  within  each  type.  Vertical 
lines  on  the  chart  denote  different  average  times  of  repair.  The  average 
bench  time  is  shown  as  11  hours,  while  the  average  unscheduled  and 
PMI  bench  times  were  5  hours  and  13  hours,  respectively.  The 
distribution  of  times  shows  how  using  the  average  can  mask  real 
variations  in  processes  and  between  repair  types. 
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Repair  Comparison  Between  MAJCOMs 
Allows  Process  Benchmarking 


184  long-pod  bench  times,  all  bases,  January  1997 — Jub  1999 
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This  chart  shows  percentiles  and  averages  of  bench  times  for  each  repair 
type  comparing  Air  National  Guard  (ANG)  to  active  bases.  In  this 
example,  the  guard  has  lower  bench  times  on  periodic  maintenance 
inspections  (PMIs)  and  time  compliance  technical  orders  (TCTOs),  but 
slightly  higher  bench  times  on  unscheduled  repairs.  The  guard  also  has 
less  variation  in  bench  times  for  all  maintenance  actions.  The  75th  and 
95th  percentile  bars  show  that  the  higher  distribution  of  times  skews  the 
average  well  above  the  median  value. 

A  note  on  this  chart  is  that  on  TCTOs,  30  percent  of  the  ACC  entries  were 
zero  and  46  percent  were  missing  the  bench  time,  whereas  45  percent  of 
the  ANG  entries  were  zero  and  30  percent  were  missing  the  bench  time. 
Data  such  as  this  can  be  used  to  compare  guard  and  active  processes  and 
their  respective  performance.  This  can  be  used  to  benchmark  existing 
processes  if  one  shows  superior  performance.  One  aspect  of  the  pod 
repair  process  we  will  consider  is  the  difference,  if  any,  between 
employing  guard  and  active  component  workers  manning  the  shops. 
Potential  savings  in  time  and  money  will  be  factored  into  the  ECM 
support  options  analysis. 

The  differences  in  bench  times  between  ACC  and  ANG  could  be  due  to 
several  factors.  First,  guard  units  tend  to  have  more  experienced  pod 
maintainers  than  active  bases.  Thus,  they  could  be  able  to  diagnose  and 
repair  faults  more  easily  than  units  with  less  experienced  personnel. 
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Also,  guard  pod  maintainers  usually  stay  at  the  same  base  for  many 
years  up  to  10  or  20  in  some  cases — so  they  have  extensive  experience 
with  their  base's  pods.  Special  knowledge  of  their  pods  could  also  help 
them  repair  more  efficiently.  The  significant  gap  in  PMI  time  could  also 
be  explained  by  more  experience,  as  that  process  would  become  highly 
routine  after  many  years. 


Trend  Analysis  Can  Be  a  Useful  Operational 
Analytical  Tool 


The  chart  above  shows  the  monthly  percentiles  and  averages  of  184  long- 
pod  bench  times  between  January  1997  and  June  1999.  Although  the 
average  time  hovers  around  20  hours,  the  median  time  (time  below 
which  half  of  all  entries  fell)  is  closer  to  10  hours,  and  the  rest  of  the 
entries  were  much  higher  than  that.  Again,  the  variation  in  the  process 
can  be  seen  with  this  type  of  data. 

An  operational  use  of  this  data  involves  charting  each  base's  and 
MAJCOM's  bench  times  in  the  same  or  a  similar  format.  This  allows  for 
performance  comparison  that  could  lead  to  important  benchmarking.  If 
one  base  shows  significant  improvements  or  simply  better  performance, 
other  bases  could  imitate  its  policies  and  procedures  to  accomplish 
similar  improvements.  This  kind  of  data  display  could  allow  the  Air 
Force  to  take  advantage  of  better  practices  that  already  exist  within  the 
force. 
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The  first  recommendation  for  repair  data  would  be  to  track  and  display 
bench  times  in  addition  to  time  to  repair.  Bench  time  is  the  clock  time  the 
pod  actually  spends  on  the  shop  bench,  whereas  time  to  repair  represents 
the  total  man-hours  spent  on  the  repair.  Adding  this  metric  could  make 
repair  measurements  more  robust  and  add  a  dimension  to  analysis  and 
operational  performance  monitoring. 

Also,  monthly  or  quarterly  repair  data  could  be  displayed  in  chart  form. 
Currently,  RAMPOD  shows  a  snapshot  of  MTTR  consisting  of  either  the 
total  history  or  the  last  year  of  data;  it  does  not  display  particular  past 
months  or  years.  Variations  and  trends  can  thus  be  masked  by  averaging 
data  over  large  periods.  If  RAMPOD  data  were  to  be  used  as  a 
diagnostic  of  a  process — in  this  case  the  repair  process — the  time  periods 
should  be  small  enough  to  yield  quick  feedback  on  process  performance. 
Observing  performance  changes  over  shorter  periods  would  allow  one  to 
respond  to  problems  more  quickly.  Again,  a  month  could  be  an 
appropriate  time  period  across  which  to  display  repair  data.  As 
mentioned  in  the  Removals  section,  data  collected  at  the  beginning  of  the 
program  was  less  consistent  and  could  compromise  the  accuracy  of  total 
history  statistics. 

Another  possible  improvement,  showing  percentiles  and/or 
distributions  of  repair  times,  would  reflect  the  range  of  bench  times.  As 
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mentioned  earlier,  showing  process  variation  contributes  to  operational 
monitoring  and  process  modeling. 

Several  of  the  graphical  displays  discussed  in  this  report  are  similar  to 
the  tools  used  in  the  Army's  Velocity  Management10  program,  which 
focuses  on  improving  logistics  processes.  Velocity  Management  has  used 
the  "Define-Measure-Improve"  methodology  to  lead  continuous 
improvement  efforts.11  RAMPOD's  current  display  could  be  effectively 
adapted  to  support  this  type  of  initiative. 

Continuing  to  display  guard  and  active  data  separately  for  comparison 
opens  up  an  opportunity  to  observe  and  leverage  different  processes. 
Again,  benchmarking  between  the  two  organizations  or  different  bases 
could  leverage  effective  practices  already  in  use.  Appropriate 
comparisons  should  be  made,  however.  Although  different  types  of 
pods  should  experience  different  repair  times,  measures  such  as  AWP  or 
AWM  may  be  comparable  across  bases  that  have  different  MDSs  because 
of  the  processes  they  represent.  Levels  and  types  of  comparisons  in  the 
display  of  data  should  thus  be  carefully  considered. 


l0For  more  background  on  this  work,  see  Dumond,  Eden,  and  Folkeson  (1995). 
nFor  a  detailed  discussion  of  Velocity  Management's  use  of  graphical  tools  and 
successful  process  improvements,  see  Wang  (2000). 
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Summary 


Recommendations 

Benefits 

•  Show  data  graphically  to 
support  entire  process 

•  Facilitate  performance 
reporting 

•  Separate  queues  and  repair 
types 

•  Differentiate  processes 

•  Identify  problems 

•  Show  range  of  data  by  using 
distributions 

•  More  accurately  portray 
population  of  pods 

•  Show  trends 

•  Observe  performance  changes 
over  time 

•  Show  MAJCOM  and  base 
comparisons 

•  Leverage  different  processes 
by  benchmarking 
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SUMMARY  OF  RECOMMENDATIONS 

In  summary,  there  are  several  improvement  opportunities  for  the 
RAMPOD  system.  Graphically  showing  data  to  support  the  complete 
pod  removal  and  repair  process,  including  removal  rates,  time  AWM, 
time  AWP,  and  repairs  themselves,  can  move  RAMPOD  toward  being  a 
self-contained  analytical  tool  for  pods.  Using  percentiles  and  /  or 
distributions  can  make  RAMPOD  analysis  more  accurate  and  robust. 
Breaking  out  parts  of  the  process  that  are  now  lumped  together  can 
expose  new  opportunities  for  improvement.  Also,  distinguishing 
between  MAJCOMs  in  displayed  data  can  create  opportunities  to 
benchmark  processes  for  further  improvement.  Finally,  using  trend 
analysis,  as  described  earlier,  can  help  operationally  by  signaling 
performance  degradation  as  well  as  strategically  by  helping  predict 
future  availability  or  requirements. 

These  suggestions  can  make  RAMPOD  more  complete  and  accurate  and 
can  therefore  render  it  more  useful  to  the  Air  Force.  RAMPOD  has  much 
untapped  potential  as  both  a  strategic  and  an  operational  tool  for  the  Air 
Force,  and  its  apparent  value  as  an  asset  could  be  greatly  increased 
through  their  implementation. 
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